Method and system for performing image registration

ABSTRACT

A method for performing image registration is provided. The method comprises obtaining a reference image dataset and a target image dataset and defining an image mask for a region of interest in the reference image dataset. The method further comprises registering a corresponding region of interest in the target image dataset with the image mask, using a similarity metric, wherein the similarity metric is computed based on one or more voxels in the region of interest defined by the image mask.

BACKGROUND

The invention relates generally to the field of image registration, andmore particularly to a method and system for efficiently registeringimages obtained via a plurality of imaging modalities.

Image registration refers to the process of finding a correspondencebetween the contents of two or more images. In particular, imageregistration refers to a process of finding a geometric transform thatnon-ambiguously links locations and orientations of objects or parts ofobjects in different images.

Image registration finds wide application in medical imaging, videomotion analysis, remote sensing, security and surveillance applications.In the field of medical imaging, a patient is generally subjected tonumerous scans over a number of imaging sessions. These scanned images(such as, for example, of a body part) may be obtained either temporallyfrom the same imaging modality or system or may be captured viadifferent imaging modalities, such as, for example, X-ray imagingsystems, magnetic resonance (MR) imaging systems, computed tomography(CT) imaging systems, ultrasound imaging systems, positron emissiontomography (PET) imaging systems and so forth. For example, PET imagingsystems and single photon emission computed tomography (SPECT) imagingsystems may be used to obtain functional body images which providephysiological information, while CT imaging systems and MR imagingsystems may be used to acquire structural images of the body whichprovide anatomic maps of the body.

As will be appreciated by those skilled in the art, the use of differentimaging modalities generates image data sets with complementaryinformation. Hardware based registration techniques are typically usefulfor performing multi-modality imaging of static structures. However, forthe imaging of dynamic structures, such as the heart, software basedregistration is additionally required to ensure a quality match. Forexample, in the diagnosis of cardio-vascular diseases for a patient, itmay be necessary to jointly visualize and correlate coronary vasculatureobtained from a CT imaging system with functional information obtainedfrom a PET/SPECT imaging system. However, the image acquisition ofdynamic structures using different modalities often has different scandurations and scan phases thereby, producing dissimilar informationcontent. Further, large field-of-view (FOV) differences and varyingresolutions between different imaging modalities may prevent theaccurate correlation of these images resulting in inaccurate diagnosisof patient information.

It would be desirable to develop a technique for efficiently andaccurately registering images obtained via a plurality of imagingmodalities. In addition, it would be desirable to jointly visualizeimage data sets obtained from different imaging modalities by reliablycoalescing the image data sets, to facilitate the generation of acomposite, overlapping image that may include additional clinicalinformation, which may not be apparent in each of the individual imagedata sets.

Brief Description

In one embodiment, a method for performing image registration isprovided. The method comprises obtaining a reference image dataset and atarget image dataset and defining an image mask for a region of interestin the reference image dataset. The method further comprises registeringa corresponding region of interest in the target image dataset with theimage mask, using a similarity metric, wherein the similarity metric iscomputed based on one or more voxels in the region of interest definedby the image mask.

In another embodiment, an imaging system is provided. The imaging systemis configured to obtain a reference image dataset and a target imagedataset. A processing module is operationally coupled to the imagingsystem. The processing module is configured to define an image mask fora region of interest in the reference image dataset. The processingmodule is further configured to register a corresponding region ofinterest in the target image dataset with the image mask, using asimilarity metric, wherein the similarity metric is computed based onone or more voxels in the region of interest defined by the image mask.

DRAWINGS

These and other features, aspects, and advantages of the presentinvention will become better understood when the following detaileddescription is read with reference to the accompanying drawings in whichlike characters represent like parts throughout the drawings, wherein:

FIG. 1 is a block diagram of an exemplary imaging system, in accordancewith one embodiment of the invention;

FIG. 2 is high-level process for performing image registration based onan image mask, using the imaging system shown in FIG. 1, in accordancewith one embodiment of the present invention.

FIG. 3( a) is an illustration of a reference image dataset correspondingto an axial slice of the heart;

FIG. 3( b) is an illustration of an image mask defined for a region ofinterest in the reference image dataset shown in FIG. 3( a);

FIG. 4( a) is an illustration of a plurality of voxels present in animage volume;

FIG. 4( b) is an illustration of a plurality of voxels present in aregion of interest defined by the image mask; and

FIG. 5 is a flowchart, illustrating process steps for performing imageregistration, using an image mask, in accordance with one embodiment ofthe invention.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of an exemplary imaging system, in accordancewith one embodiment of the invention. The imaging system 10 may beconfigured to facilitate acquisition of image data from a patient (notshown) via a plurality of image acquisition systems. Although, theillustrated embodiments are described in the context of a medicalimaging system, it will be appreciated that the imaging system 10 mayalso be used in industrial applications, such, for example, baggagescanning applications, and other security and surveillance applications.

Referring to FIG. 1, the imaging system 10 is illustrated as including afirst image acquisition system 12, a second image acquisition system 14and an N^(th) image acquisition system 16. It may be noted that thefirst image acquisition system 12 may be configured to obtain a firstimage dataset representative of the patient under observation. In asimilar fashion, the second image acquisition system 14 may beconfigured to facilitate acquisition of a second image datasetassociated with the same patient, while the N^(th) image acquisitionsystem 16 may be configured to facilitate acquisition of an N^(th) imagedataset from the same patient.

In one embodiment, the imaging system 10 is representative of amulti-modality imaging system. In other words, a variety of imageacquisition systems may be employed to obtain image data representativeof the same patient. More particularly, in certain embodiments each ofthe first image acquisition system 12, the second image acquisitionsystem 14 and the N^(th) image acquisition system 16 may include a CTimaging system, a PET imaging system, an ultrasound imaging system, anX-ray imaging system, an MR imaging system, an optical imaging system orcombinations thereof. For example, in one embodiment, the first imageacquisition system 12 may include a CT imaging system, while the secondimage acquisition system 14 may include a PET imaging system and theN^(th) image acquisition system 16 may include an ultrasound imagingsystem. It may be noted that it is desirable to ensure similardimensionality of the various image acquisition systems in themulti-modality imaging system 10. In other words, in one embodiment, itis desirable that in the multi-modality imaging system 10, each of thevarious image acquisition systems 12, 14, 16 includes a two-dimensionalimage acquisition system. Alternatively, in certain other embodiments,the multi-modality imaging system 10 entails use of three-dimensionalimage acquisition systems 12, 14, 16. Accordingly, in the multi-modalityimaging system 10, a plurality of images of the same patient may beobtained via the various image acquisition systems 12, 14 and 16.

Further, in certain other embodiments, the imaging system 10 may includeone image acquisition system, such as the first image acquisition system12. In other words, the imaging system 10 may include a single modalityimaging system. For example, the imaging system 10 may include only oneimage acquisition system 12, such as a CT imaging system. In thisembodiment, a plurality of images, such as a plurality of scans takenover a period of time, of the same patient may be obtained by the sameimage acquisition system 12.

The plurality of image datasets representative of the patient that havebeen obtained either by a single modality imaging system or by differentimage acquisition modalities may then be merged to obtain a combinedimage. As will be appreciated by those skilled in the art, imagingmodalities such as PET imaging systems and single photon emissioncomputed tomography (SPECT) imaging systems may be employed to obtainfunctional body images which provide physiological information, whileimaging modalities such as CT imaging systems and MR imaging systems maybe used to acquire structural images of the body which provide anatomicmaps of the body. As mentioned above, these different imaging techniquesare known to provide image data sets with complementary and occasionallyconflicting information regarding the body. In accordance withembodiments of the present invention, the image datasets obtained fromdifferent imaging modalities are combined to generate a composite,overlapping image that may include additional clinical information,which may not be apparent in each of the individual image data sets.More particularly, the composite image facilitates clinicians to obtaininformation regarding shape, size and spatial relationship betweenanatomical structures and any pathology, if present.

Referring to FIG. 1 again, the plurality of acquired image datasets maybe registered to generate a composite image to facilitate clinicians tocompare or integrate data representative of the patient obtained fromdifferent measurements. In one embodiment, one or more imageregistration techniques may be utilized to coalesce the plurality ofimage sets obtained by the imaging system 10 via the processing module18. In the example illustrated in FIG. 1, the processing module 18 isoperatively coupled to the image acquisition systems 12, 14, 16 and isconfigured to facilitate the registration of the plurality of acquiredimage datasets to generate a composite, registered image. In aparticular embodiment, and as will be described in greater detail below,the processing module 18 is configured to define an image mask for aregion of interest in a reference image dataset. The processing module18 is further configured to register a corresponding region of interestin a target image dataset with the image mask, using a similaritymetric, wherein the similarity metric is computed based on one or morevoxels in the region of interest defined by the image mask. Theprocessing module 18 may be accessed and/or operated via an operatorconsole 20. The operator console 20 may also be employed to facilitatethe display of the composite registered image generated by theprocessing module 18, such as on a display 22 and/or a printer 24. Forexample, an operator may use the operator console 20 to designate themanner in which the composite image is visualized on the display 22.Operator console 20 may be equipped to include a user interface that isresponsive to user inputs for visualization and display preferences.

FIG. 2 is high-level process for performing image registration based onan image mask, using the imaging system shown in FIG. 1, in accordancewith one embodiment of the present invention. As mentioned above, imageregistration refers to a process of transforming different imagedatasets into a common coordinate system. More particularly, the processof image registration involves finding one or more suitabletransformations that may be employed to transform the image datasetsunder study to a common coordinate system.

Referring to FIG. 2, the image datasets include a reference imagedataset 26, and at least one target image dataset 28. As used herein, a“reference image dataset” refers to an image that is unchanged andthereby used as a reference. It may be noted that the terms referenceimage, original image, source image and fixed image may be usedinterchangeably. Additionally, the other acquired images to be mappedonto the reference image dataset may be referred to as “target” imagedatasets. In other words, the target image dataset embodies the imagethat is geometrically transformed to spatially align with the referenceimage dataset. It may also be noted that the terms target image, movingimage, sensed image and floating image may be used interchangeably. Inone embodiment, the reference image dataset 26 and a target imagedataset 28 correspond to image datasets representative of a patientacquired via different imaging modalities or alternatively, imagedatasets acquired via a same modality but at different instants of time.In a particular embodiment, the reference image dataset 26 includesanatomical information acquired using a computed tomography (CT) imagingsystem and the target image dataset 28 comprises functional informationacquired using a positron emission tomography (PET) imaging system. Asused herein, “anatomical information” may include, for example ananatomic landscape indicative of distinct anatomical regions in apatient and “functional information” may include, for example,physiological information associated with a patient.

In accordance with embodiments of the present invention, an image mask30 is further defined for a region of interest in the reference imagedataset 26. In one embodiment, the image mask 30 corresponds to a massof uniform intensity over a region of interest in the reference imagedataset 26. In another embodiment, the image mask 30 may be anapproximation of a region of interest akin to a large biopsy of tissue.Further, the region of interest defined by image mask 30 may also extendto a neighboring (for example, a dilated or expanded) region, so thatboundary information associated with the region of interest may beincluded in the image mask.

In a particular embodiment, the image mask 30 is a cardiac mask. FIG. 3(a) is an illustration of a reference image dataset corresponding to anaxial slice of the heart. FIG. 3( b) is an illustration of an image maskdefined for a region of interest in the reference image dataset shown inFIG. 3( a). In one embodiment, the image mask 30 may be defined byautomatically segmenting the region of interest in the reference imagedataset 26. As will be appreciated by those skilled in the art,segmentation is a process of selecting regions of interest that are asubset of a larger image volume. The segmentation into a region ofinterest may be based upon apriori information, such as, for example,anatomical information associated with the region of interest. Asdescribed above, an image mask is substantially automatically defined.Further, the segmentation may be performed using automatic orsemi-automatic techniques. In another embodiment, the image mask may becreated manually from the reference image dataset. In one embodiment, auser may also be permitted to define the image mask for a region ofinterest in the reference image dataset 26, using for example a userinterface, which may be part of operator console 20 of FIG. 1.

In another embodiment, the image mask may also be createdsemi-automatically, using an image reference point. As will beappreciated by those skilled in the art, during imaging, a referencepoint of interest is often stored with an image. For example, in acardiac image scan, the reference point of interest may include theco-ordinates of the apex of the image. An image mask may be createdautomatically from this reference point of interest, based on thedimensions of a typical heart. In yet another embodiment, the image maskmay be created using an atlas based localization technique, wherein thereference image dataset is automatically registered using relevant atlasdata. In particular, an atlas image of the anatomy of interest is usedin order to automatically generate an image mask from the referenceimage. The atlas image is registered with the reference image to obtaina registered atlas image. The registered atlas image may then be used asan image mask.

Referring to FIG. 2 again, the target image dataset 28 may begeometrically transformed to spatially align with the reference imagedataset 26, using a transformation model 38. The transformation model 38locates a plurality of reference image dataset coordinates in the targetimage dataset 28 and aligns a plurality of pixel correspondences for aregion of interest between the image datasets, to generate a registeredimage. As used herein, “pixel correspondences” refer to the associationof two positions, one from each image dataset that reference anidentical position on a feature/region of interest or object beingimaged. The transformation component 38 is further configured to apply atransform to register the region of interest in the target image dataset28 with the image mask 30 in the reference image dataset 26. Thetransform may include, a rigid transform or a non-rigid transform. Rigidtransforms may include, for example, translations, rotations, scaling,skew or combinations thereof. Non-rigid transforms may includedeformation fields generated by typical methods, for example, finiteelement modeling (FEM), B-splines, optic flow based (Daemon's) method,diffusion based methods, or level-set based methods.

A similarity metric 32 quantifies the degree of correspondence betweenthe pixels or voxels in both the reference and target image datasetsthat is achieved by the transformation model 38. The similarity metric32 may include, but is not limited to, a contrast measure, minimizingmeans-squared error, correlation ratio, ratio image uniformity (RIU),partitioned intensity uniformity (PIU), mutual information (MI),normalized mutual information (NMI), joint histogram, or joint entropy,for example. As will be appreciated by those skilled in the art, sincethe computation of the similarity metric for reference and target imagesis computationally intensive, desirably a sample or percentage of voxelsare selected randomly for metric computation, wherein the selectedvoxels are assumed to be statistically representative of the entireimage volume. FIG. 4( a) is an illustration of a plurality of voxelspresent in an image volume.

In accordance with embodiments of the present invention, the number ofvoxels used for metric computation is restricted to the region ofinterest defined by the image mask 30 in the reference image dataset. Ina particular embodiment, the similarity metric 32 is computed based on aplurality of voxels within the region of interest defined by the imagemask 30 as shown in FIG. 4( b). In particular, since the computation ofthe similarity metric is targeted to only the portion of the imagevolume defined by the image mask, the computation time is significantlyreduced.

Accordingly, the use of the image mask results in a robust metriccomputation process and improves the accuracy of the image registrationprocess and results in better alignment of images. In particular, thecomputation of the similarity metric is performed by sampling the voxelsin the region of interest defined by the image mask. In one embodiment,all the voxels in the region of interest may be sampled to compute thesimilarity metric. In another embodiment, a selected percentage ofvoxels in the region of interest may be sampled, wherein the voxelsamples may be chosen randomly. In yet another embodiment, a sequentialuniform sampling of the voxels in the region of interest may beperformed, by choosing voxels samples uniformly. For example, every nthvoxel in a region of interest may be chosen, to generate a 100/n %sampling of the voxels. Further, in accordance with one embodiment, theanatomical information in the image mask may be weighted, whereindifferent sub-regions in the image mask may be assigned differentweights. Further, these weights may be used in the computation of thesimilarity metric of the voxels in a particular sub-region. In aparticular embodiment, the weights may include information about therelevance of each voxel for a particular application, and thisinformation may be used in voxel selection and similarity metriccomputation. For example, in a cardiac application, the vessels may beassigned a higher weight compared to a myocardium wall tissue duringcomputation of the similarity metric for comparing images.

In another embodiment, the process of image registration using an imagemask may be viewed as a multi-scale process, that is, the registrationprocess begins at a coarsest copy of the images, and the results ofregistration is re-used for registration at the next finer level, tillthe final image resolution is reached. This method often gives betterand faster results in cases when images are grossly mis-aligned.Further, the image mask generated in accordance with embodiments of thepresent invention may be used to enable the joint visualization ofcoronary vasculature obtained from a high resolution CT system alongwith functional information from a PET/SPECT acquisition. The compositeregistered image generated may be used to aid diagnostic assessment andcorrelation of myocardial defects (such as for example, infact andreversible tissue and cardiovascular disease) for a patient.

Referring to FIG. 2 again, an optimizer 36 may be used to maximize theimage similarity between the image datasets, by selecting atransformation parameter that optimizes the similarity metric 32.Further, an interpolator 34 may be used to approximate the set of pixelcorrespondences between the image datasets. As will be appreciated bythose skilled in the art, it may be desirable to optimize a measureassociated with the similarity metric. Accordingly, a suitabletransformation parameter may be selected such that the measureassociated with the similarity metric is optimized. This transformationparameter may then be employed to transform the target image dataset 28to the reference image dataset 26 to generate a registered image.

FIG. 5 a flowchart, illustrating process steps for performing imageregistration using an image mask, in accordance with one embodiment ofthe invention. In step 52, a reference image dataset 26 and a targetimage dataset 28 are obtained. In one embodiment, the reference imagedataset 26 comprises anatomical information acquired using a computedtomography (CT) imaging modality and the target image dataset 28comprises functional information acquired using a positron emissiontomography (PET) imaging modality. In step 54, an image mask 30 isdefined for a region of interest in the reference image dataset 26. Inone embodiment, the image mask 30 is a cardiac mask. In step 56, acorresponding region of interest in the target image dataset 28 isregistered with the image mask 30, using a similarity metric 32 togenerate a registered image. The registered image may be furtherdisplayed to a user. In one embodiment, and as described above, thesimilarity metric 32 is computed based on one or more voxels in theregion of interest defined by the image mask 30 in the reference imagedataset 26.

While only certain features of the invention have been illustrated anddescribed herein, many modifications and changes will occur to thoseskilled in the art. It is, therefore, to be understood that the appendedclaims are intended to cover all such modifications and changes as fallwithin the true spirit of the invention.

1. A method for performing image registration, comprising the steps of:obtaining a reference image dataset and at least one other target imagedataset; defining an image mask for a region of interest in thereference image dataset; and registering a corresponding region ofinterest in the at least one other target image dataset with the imagemask, using a similarity metric, wherein the similarity metric iscomputed based on one or more voxels in the region of interest definedby the image mask.
 2. The method of claim 1, wherein the reference imagedataset and the at least one other target image dataset is acquired viaan imaging system, wherein the imaging system comprises at least one ofa computed tomography (CT) imaging system, a positron emissiontomography (PET) imaging system, a magnetic resonance (MR) imagingsystem, an X-ray imaging system, an ultrasound imaging system, orcombinations thereof.
 3. The method of claim 2, wherein the referenceimage dataset is acquired via a first imaging modality and the at leastone other target image dataset is acquired via a second imagingmodality, where the second imaging modality is different from the firstimaging modality.
 4. The method of claim 2, wherein the reference imagedataset and the at least one other target image dataset are acquired viaa same imaging modality at different points in time.
 5. The method ofclaim 2, wherein the reference image dataset and the at least one othertarget image dataset comprise at least one of anatomical information andfunctional information.
 6. The method of claim 5, wherein the referenceimage dataset comprises anatomical information acquired using a computedtomography (CT) imaging modality and the at least one other target imagedataset comprises functional information acquired using a positronemission tomography (PET) imaging modality.
 7. The method of claim 1,wherein computing the similarity metric comprises sampling one or moreof the voxels in the region of interest defined by the image mask. 8.The method of claim 1, wherein computing the similarity metric comprisesweighting one or more sub-regions in the region of interest defined bythe image mask, based on anatomical information comprising the imagemask.
 9. The method of claim 1, comprising automatically segmenting theregion of interest in the reference image dataset to define the imagemask.
 10. The method of claim 1, comprising automatically registeringthe reference image dataset using an atlas-based localization technique,to define the image mask.
 11. The method of claim 1, comprisingpermitting a user to define the image mask for the region of interest inthe reference image dataset.
 12. The method of claim 1, wherein thesimilarity metric comprises at least one of mutual information (MI),contrast measure, minimizing means-squared error, correlation ratio,ratio image uniformity (RIU), normalized mutual information (NMI), jointhistogram, and joint entropy.
 13. The method of claim 1, furthercomprising applying at least one of a rigid transform and a non-rigidtransform to register the region of interest in the at least one othertarget image dataset with the image mask in the reference image dataset.14. The method of claim 1, further comprising generating a registeredimage.
 15. The method of claim 10, further displaying the registeredimage to a user.
 16. A system, comprising: at least one imaging systemconfigured to obtain a reference image dataset and at least one othertarget image dataset; and a processing module operationally coupled tothe at least one imaging system and configured to define an image maskfor a region of interest in the reference image dataset and register acorresponding region of interest in the at least one other target imagedataset with the image mask, using a similarity metric, wherein thesimilarity metric is computed based on one or more voxels in the regionof interest defined by the image mask.
 17. The system of claim 16,wherein the imaging system comprises at least one of a computedtomography (CT) imaging system, a positron emission tomography (PET)imaging system, a magnetic resonance (MR) imaging system, an X-rayimaging system, an ultrasound imaging system, or combinations thereof.18. The system of claim 17, wherein the reference image dataset isacquired via a first imaging modality and the at least one other targetimage dataset is acquired via a second imaging modality, where thesecond imaging modality is different from the first imaging modality.19. The system of claim 17, wherein the reference image dataset and theat least one other target image dataset are acquired via the sameimaging modality at different points in time.
 20. The system of claim17, wherein the reference image dataset and the at least one othertarget image dataset comprise at least one of anatomical information andfunctional information.
 21. The system of claim 20, wherein thereference image dataset comprises anatomical information acquired usinga computed tomography (CT) imaging modality and the at least one othertarget image dataset comprises functional information acquired using apositron emission tomography (PET) imaging modality.
 22. The system ofclaim 16, wherein the processing module is configured to automaticallysegment the region of interest in the reference image dataset to definethe image mask.
 23. The system of claim 16, wherein the processingmodule is configured to automatically register the reference imagedataset using an atlas-based localization technique, to define the imagemask.
 24. The system of claim 16, further comprising permitting a userto define the image mask for the region of interest in the referenceimage dataset.
 25. The system of claim 16, wherein the processing moduleis configured to compute the similarity metric by weighting one or moresub-regions in the region of interest defined by the image mask, basedon anatomical information comprising the image mask.
 26. The system ofclaim 16, wherein the processing module is configured to apply at leastone of a rigid transform and a non-rigid transform to register theregion of interest in the at least one other target image dataset withthe image mask in the reference image dataset.
 27. The system of claim16, wherein the processing module is further configured to generate aregistered image.
 28. The system of claim 16, wherein the system furthercomprises a display module configured to display the generatedregistered image.